System for and method of configurable line scan array imaging

ABSTRACT

Disclosed are image data acquisition methods and systems that utilizes selective temporal co-adding of detector integration samples to construct improved high-resolution output imagery for arrays with selectable line rates. Configurable TDI arrays are used to construct output imagery of various resolutions dependent upon array commanding, the acquisition geometry, and temporal sampling. The image acquisition techniques may be applied to any optical sensor system and to optical systems with multiple sensors at various relative rotations which enable simultaneous image acquisitions of two or more sensors. Acquired image data may be up-sampled onto a multitude of image grids of various resolution.

BACKGROUND 1. Technical Field

The present disclosure relates generally to image acquisition and, inparticular, to systems and methods for constructing enhanced resolutionoutput image using configurable time delay and integrate (TDI) sensorarrays. Embodiments pertain to methods and integrated circuits foroptical imaging including read-out integrated circuits (ROICs) with TDIcapability.

2. Discussion of Related Art

Solid-state detectors such as charge coupled devices (CCD) or CMOSActive Pixel Sensors are widely used in a variety of imagingapplications. Typically, the sampling elements (detectors associatedwith unit cells) are arranged in rows and columns to convert theelectromagnetic radiation from a scene into charges that are convertedinto electrical signals. A linear array consists of only one row ofdetectors (one dimensional, 1D) while an area array consists of an arrayof detectors with multiple rows and columns (two dimensional, 2D).

If an imaged scene is moving with respect to the detector and thecorresponding movement during the integration period on the detector isconsiderable with respect to the pixel pitch, the resulting image willbe blurred. In many imaging applications, the image scene moves relativeto the detector with a constant or predictable velocity. A well-knowntechnique in satellite or aircraft imaging applications is push-broomimaging, wherein a camera images the ground scene. In such applications,a 1D array can be used to generate 2D images by repeatedly exposing andintegrating on the single row of pixels while moving the detector in adirection orthogonal to the long dimension of the array. The directionof the motion is called “in-scan”, while the direction orthogonal tothis motion direction is called “cross-scan”.

In line scan applications where the light level is low, or where therelative speed of the movement is large, TDI image sensors comprised of2D pixel arrays are typically used, wherein the pixel signals deliveredby the pixels of the same column are delayed and added synchronouslywith the optical scanning. Thus, the light from a given point in thescene impinges successively on each pixel of the given correspondingcolumn. As the light of the scene impinges on each row in succession,the signals from each of the rows are added to increase the final SNR.The TDI principle has typically been addressed with CCD sensors, wherethe TDI functionality is more or less intrinsically available byshifting the charge packets along the CCD synchronously with the movingimage. Exemplary array control through commanding is disclosed by U.S.Pat. No. 8,629,387 to Pflibsen et al., entitled “Multi-layer Sensor ChipAssembly and Method for Imaging Generating Image Data with a Frame-SumMode and a Time Delay Integration Mode”, the contents of which arehereby incorporated by reference.

System sensitivity may be increased by increasing the number of TDIpixels in a scanning focal plane array (FPA), but as the number of TDIdetector elements increase, it becomes increasingly difficult torestrict an object in the scene to travel within a single TDI row. Lensbarrel distortion and scan geometry are some of the common factors thatmay move a given scene portion off of a given detector bank, resultingin a blurry picture and relatively little sensitivity improvement fromthe TDI operation.

In current line scanning arrays, scan smear increases for reduced linerate data acquisition from the FPA's data registers, decreasing thevalue of the imaging system modulation transfer function (MTF).Improving the image quality resolution and in-scan MTF would, thus, beuseful. Reconstruction of a high-resolution image from a sequence oflower resolution images is a way to increase the effective spatialresolution of a camera capturing conventional images. U.S. Pat. No.8,558,899 to Thomas J. Grycewicz, entitled “System and Method forSuper-Resolution Digital Time Delay and Integrate (TDI) ImageProcessing”, which is incorporated by reference in its entirety,discloses processing oversampled imagery to determine scan smear afterthe imaging system collects the imagery. Another technique is describedin U.S. Pat. No. 8,463,078 to Goodnough et al., incorporated byreference herein, wherein images from multiple arrays are processed toimprove image quality. “Image Super-Resolution for Line Scan Camerasbased on a Time Delay Super-Resolution Principle”, Daniel Soukup, AIT,2009, discloses an impractical asynchronous oversampling of array data,followed by complex processing, in order to improve smear fornon-existent array types. While these devices may fulfill theirrespective, particular objectives, they do not disclose implementationsof reduced in-scan smear data generation or improvements based on singleimage acquisition.

Current data acquisition techniques for TDI arrays are limited toscanning normal to the cross-scan array dimension. To produce a linerate which is less than or equal to the detector's readout rate, adetector is summed multiple times at the array readout rate to yield anaccumulated value, as an input to the processing that is in-scan smearedacross approximately two projected detector field of view (PDFOV) areasof the area being imaged. No usable TDI imagery is possible for scanrates greater than the detector's readout rate, or if the scan geometryis not normal to the array's cross-scan axis. In current field offset,non-parallel multi-sensor platforms, only a single sensor's data isacquired per collect, as the other TDI array sensors are limited toorthogonal scanning. Thus, no usable imagery from secondary sensors intheir standard operating modes can be obtained. Thus, it would also beuseful to be able to configure a TDI array in order to acquire imagedata for various conditions in order to enable creation of selectablehigher resolution imagery, to acquire secondary mission imagerysimultaneously with multiple sensors, over larger area collects inreduced time, and for any scan geometry and rate.

SUMMARY

In accordance with an embodiment, a method is provided for generatingreduced in-scan blur image data using an imaging device including acontroller, a digital memory structure having a plurality of storagelocations, where each storage location is configured to store a digitalvalue and is individually addressable by the controller. The imagingdevice may also include an array of unit cells (e.g., charge coupleddevices) configured to store charge based on detected photons, whereeach of the unit cells is associated with a corresponding storagelocation and includes a time delay and integrate (TDI) detector. Theimaging device has a readout rate that is greater than a line rate,where the readout rate defines readout-clock times for implementingcommands to enable discrete, temporary operating modes. The method mayinclude commencing with selecting shift mode readout-clock times andaccumulate mode readout-clock times, where the accumulate modereadout-clock times representing times during which scene data should beaccumulated from one or more of the unit cells. For each of the selectedaccumulate mode readout-clock times and for each of a set of the one ormore unit cells of the array, the controller causes accumulation over anintegration time of detector charge associated with a projected detectorfield of view (PDFOV) of the scene to obtain a detector co-add datasample, reading of the detector co-add data sample from the unit cell,and adding of the detector co-add data sample to existing detectorco-add scene data in the storage location corresponding to the unitcell. For each of the selected shift mode readout-clock times, and foreach unit cell in the set of one or more unit cells, the controllercauses the shifting of existing detector co-add scene data stored in aplurality of the storage locations corresponding to the set of unitcells to a plurality of shifted storage locations by a number of rowsthat is based on a rate of motion change of the scene in relation to thePDFOV, and outputting at the line rate from one or more of the pluralityof storage locations of the detector co-add scene data corresponding toa line of scene data, to form an enhanced image of the scene.

In other example embodiments, the line rate, integration time, and setof unit cells may each or all be separately selected (e.g., by a user,etc.). The data shift in storage locations for detector co-add scenedata may also be selectable, e.g., so as to maximize the signal range ofthe imaging device.

In another embodiment, the controller may select (and may receive as aninput) which one or more unit cells may be active during the selectedaccumulate mode readout-clock times.

The image data sets may be captured in visible and infrared wavelengthranges. The TDI detectors included in the array of unit cells may bearrayed in one or more rows. The relative position of the scene changesa fractional portion or an integral number of PDFOVs along an in-scandirection of the one or more rows of the detectors between eachsuccessive readout-clock time.

In another embodiment, a method is provided for generating reducedin-scan blur image data with an imaging device including an array ofunit cells configured to store charge based on detected photons, acontroller, and a digital memory structure having a plurality of storagelocations, each storage location for storing a digital value and beingindividually addressable by the controller, and each of the unit cellsbeing associated with a corresponding storage location. The controllerperforms the steps of determining a digital value based on a storedcharge from at least one of the unit cells, adding the determineddigital value to an existing value in a corresponding storage locationwhen operating in accumulate mode, and selectively adding the determineddigital value to an existing value in a storage location when operatingin accumulate mode, wherein during shift mode, the storage locationsshift values from adjacent storage locations and an output is generatedfrom a set of storage locations.

In another aspect, a line scanning imaging device is provided that has areadout rate that is greater than a line rate, and wherein the readoutrate defines readout clock times for selectively enabling shift and/oraccumulate modes. The imaging device may include a focal plane array ofunit cells each including a TDI detector configured to accumulate chargebased on detected photons of the PDFOV of a scene. The device may alsoinclude a digital memory structure having a plurality of storagelocations, each associated with a corresponding unit cell and configuredto store a digital value indicative of the charge and to be individuallyaddressable. The device may also include a controller configured tooperate in an accumulate mode and a shift mode, wherein, in theaccumulate mode, the controller determines the digital value based onthe stored charge from at least one of the unit cells, and selectivelyadds the at least one determined digital value to existing values in atleast one corresponding storage location, and in the shift mode, thecontroller shifts values from the at least one corresponding storagelocation to adjacent storage locations.

The controller may be further configured to select shift modereadout-clock times and accumulate mode readout-clock times, where theaccumulate mode readout-clock times represent times during which scenedata should be accumulated from one or more of the unit cells. For eachof the selected accumulate mode readout-clock times and for each of aset of the one or more unit cells, the controller may cause theaccumulation over an integration time of detector charge associated withthe PDFOV of the scene in order to obtain a detector co-add data sample,read the detector co-add data sample from the unit cell, and add thedetector co-add data sample to existing detector co-add scene data inthe storage location corresponding to the set of unit cells. For each ofthe selected shift mode readout-clock times, and for each unit cell, thecontroller may cause the accumulation over an integration time of thedetector charge associated with the PDFOV of the scene to obtain adetector co-add data sample, shifting of the existing detector co-addscene data stored in the at least one corresponding storage locations toa plurality of shifted storage locations by a number of rows that isbased on a rate of motion change of the scene in relation to the PDFOV,and outputting at the line rate from one or more of the plurality ofstorage locations of the detector co-add scene data corresponding to aline of scene data, to form an enhanced image of the scene.

The TDI detectors may be responsive to electromagnetic radiation in thevisible and infrared regions of the spectrum. The array of unit cellsmay have an imaging geometry providing that the scene is repositionedalong an in-scan direction of the array. The controller may beconfigured to select (or receive as a selection input) the line rate,integration time, and designation of which unit cells should be in theset for accumulation. The controller may be further configured to selectwhich unit cell may be active during the selected accumulate modereadout-clock times.

It is to be understood that the summary, drawings, and detaileddescription are not restrictive of the scope of the inventive conceptdescribed herein.

BRIEF DESCRIPTION OF THE FIGURES

The foregoing and other objects, features and advantages will beapparent from the following, more particular description of theembodiments, as illustrated in the accompanying figures, wherein likereference characters generally refer to identical or structurally and/orfunctionally similar parts throughout the different views. The figuresare not necessarily to scale, emphasis instead being placed uponillustrating the principles of the embodiments, wherein:

FIG. 1 is a block diagram showing an example embodiment of a digitalsynthetic super-resolution imager system;

FIG. 2 is a block diagram of an integrated circuit implementation of animager system with selectable line rate capability, in accordance withsome embodiments;

FIG. 3 is a flow diagram of a method for generating image data, inaccordance with some embodiments;

FIG. 4 is an illustration of modified commanding to produce slower linerates in an imager system;

FIG. 5 is an illustration of modified commanding to reduce Co-Add datasample collection in an imager system;

FIG. 6 is an illustration of an example projected detector field of view(PDFOV) four Co-Add data sampling integration, in accordance with someembodiments;

FIG. 7 is an illustration of an example PDFOV single Co-Add datasampling integration, in accordance with some embodiments;

FIG. 8 is a plot of scan smear MTF for an existing line scan method andfor two embodiments of the EEMTF method described herein;

FIGS. 9A and 9B are plots of in-scan signal response for four-sampleembodiments comparing results from existing approaches and EEMTFmethods;

FIG. 10 is a flow diagram of a method for processing a data setinclusive of archival, image generation, dissemination options, remoteimage generation, and remote image re-processing for enhanced resolutionimage data, in accordance with some embodiments;

FIG. 11A is an illustration of an example PDFOV single Co-Add datasampling integration, and FIGS. 11B-11F are illustrations of pixelresampling grids, in accordance with some embodiments;

FIG. 12 is an illustration depicting a multi-resolution image display,in accordance with some embodiments;

FIG. 13 are images of an original scene, the diagonally scanned sceneimage processed conventionally, and with DICE enhanced processing,according to some embodiments;

FIGS. 14A and 14B are illustrations showing, respectively, a four-sensorsystem scanned in a normal and at an angle, and a three-sensor systemscanned with one sensor normal and two sensors at angles, in accordancewith some embodiments;

FIG. 15 is an illustration of a sparse enabled configurable TDI array,in accordance with some embodiments;

FIG. 16 is an illustration of a pixel grid for a sparse enabledconfigurable TDI array for a normal scan angle and a sub-sampling for a4×PDFOV scan rate, in accordance with some embodiments; and

FIG. 17 are images of an original scene, the scanned scene imageprocessed conventionally, and in accordance with some embodiments of asecondary mission image acquisition and processing method.

DETAILED DESCRIPTION

The following discussion of embodiments of enhanced optical imagingusing configurable line scan array imaging methods and systems is merelyexemplary in nature, and is in no way intended to limit the disclosedembodiments or their applications or uses. Alternatives to theembodiments disclosed may be devised without departing from the scope ofthe disclosure. For example, several embodiments are described in termsof sequences of actions to be performed by, for example, by a processoror controller (the terms used interchangeably herein). It will berecognized that various actions described herein can alternatively beperformed by specific circuits (e.g., application specific integratedcircuits (ASICs)), by program instructions being executed by one or moredistributed processors, or by a combination thereof.

Well-known elements of technologies associated with the embodiments willnot be described in detail, or will be omitted, so as not to obscure therelevant details of the novel methods and apparatus. The word“exemplary” is used herein to mean “serving as an example, instance, orillustration.” Any embodiment described herein as “exemplary” is notnecessarily to be construed as preferred or advantageous over otherembodiments. Likewise, the term “embodiment” and the descriptivelanguage associated with each use of the term do not require that allembodiments include the discussed feature, limitation, advantage or modeof operation. As used herein, the singular forms “a”, “an” and “the” areintended to include the plural forms as well, unless the context clearlyindicates otherwise. It will be further understood that the terms“comprises”, “comprising”, “having”, “includes” and/or “including”, whenused herein, specify the presence of stated features, steps, operations,elements, and/or components, but do not preclude the presence oraddition of one or more other features, steps, operations, elements,components, and/or groups thereof. Further, the phrase “based on” isintended to mean “based, at least in part, on” unless explicitly statedotherwise.

As used herein, an “image” may be defined as a display comprising“pixels” defined as image display elements. The terms “sub-region” or“sub-frame”, used interchangeably herein, may be defined as one of a setof images that may be used to reconstruct a super-resolution image. A“pixel grid” may be defined as a map of the center of each detector inthe imaging device (e.g., focal plane array (FPA)), and an “image grid”may be defined as a two-dimensional projection of these points onto thesurface being imaged for a particular sub-region. A “detector” may bedefined as an energy gathering element. A “detector array”, or “sensor”,may be defined as an array consisting of m rows by n columns ofdetectors. A sensor “field-of-view” (FOV) may be defined as the extentof target region that can be observed by the detector array at onemoment. For a TDI array, each of the n columns may have the m rowscombined in time to reduce noise. A “projected detector FOV” (PDFOV) maybe defined as the target region observed by a single detector at onemoment. “Detector data” may be defined as the integrated energy of adetector sensing a PDFOV for a readout-clock time, a summation ofmultiple detector data samples, or a summation of a set or subset ofdetector Co-Add data. A “readout-clock time” may be defined as theshortest line output period for the sensor. A “Detector Co-Add data”sample may be defined as the detector data for a single readout-clocktime. A “data line” may be defined as detector data for the entirecross-scan extent of a detector array.

Imager System Architecture

An example embodiment of a TDI imager system 100 for enhanced linescanning imaging is shown in FIGS. 1 and 2. TDI imager system 100 mayinclude a detector array (or sensor) 102 comprised of detectors 104 andassociated unit cells 106 that are operatively and independentlyinterconnected to a controller 108 (a processor, or other computingdevice). In an example embodiment, sensor 102 is responsive toelectromagnetic radiation in the visible or infrared region of thespectrum. Sensor 102 may comprise a configurable focal plane array(FPA), and may include charge-coupled devices (CCD), such as arecommonly utilized for TDI line-scanners. FPAs are capable of providinghigh-resolution wide-area coverage and would thereby find application inair and space intelligence, surveillance, and reconnaissance. In suchapplications, FPAs can reduce the number of assets required for globalpersistent day/night surveillance and launch detection and fully utilizediffraction-limited resolution.

In an illustrative embodiment, sensor 102 detects incoming opticalradiation 110 from a scene 112 using detector photodiodes to createcurrents that are proportional to the optical radiation 110. That is,each detector 104 in the detector array 102 produces a current that isproportional to the photon flux impinging upon the detector 104. Eachdetector 104 is associated with one of the unit cells 106. Thecontroller 108 may issue commands 114 that determine from which of thedetectors 104 current is collected and stored as charge in theassociated unit cells 106, where electronics integrate the charges andproduce, via analog to digital converters (ADCs), digital numbers (DN)proportional to the total charge accumulated over the frame period.

In an example embodiment, sensor 102 may comprise a TDI imager with theTDI imaging geometry optimized as an input 116 to controller 108programmed to perform super-resolution image reconstruction processing.Controller 108 may also be programmed to implement processing tasksincluding, but not limited to, re-sampling as described herein.Controller 108 may have access to a digital memory structure 118 havinga plurality of data storage locations 120 _(a,a)-120 _(m,n) (e.g., dataregisters) individually addressable by the controller 108 and eachcorresponding to one of the unit cells 106. Controller 108 may beimplemented in a variety of ways and may take the form of a single-chipmicrocontroller, a mainframe computer, or anything in between.Controller 108 may through commands 122 be configured to initiate thetransfer of output imagery data 124 from certain storage locations 120and to control shift and accumulate modes for operating sensor 102,up/down counting, scan and line rate timing, and integration times.Controller 108 may also perform on-chip digital signal processingfunctions, such as the image enhancement processes described below.Commanding functions may be predetermined for autonomous operation, ormay comprise external control signals received within the input 116. Itshould also be appreciated that the processing functionality provided bythe controller 108 can be distributed among multiple computing devicesand/or provided at least in part via a remote communications link.

Scene 112 is shown in motion (indicated by arrows 126) relative todetector array 102. In an example embodiment, detector array 102 ismounted such that the rows of detectors 104 in the array form an angleparallel or perpendicular to the direction of scene motion. In this way,each detector 104 in the detector column is presented sequentially witha particular portion of the scene 112. The photo-charge accumulated ineach successive detector 104 during the time that the portion of thescene 112 moves over the detector 104 contributes to the respectivepixel in the captured image. Whereas in a conventional TDI imager,camera timing would be synchronized such that the scene 112 moves byexactly one pixel/detector per frame, in an example embodiment of theTDI imager system 100 described herein, scan timing is set such that thescene 112 traverses a fractional portion of a DPFOV or an integralnumber of DPFOVs between image lines. In an example embodiment,controller 108 is programmed to process image 128 by compensating forthe image motion and summing on a high-resolution image grid asdescribed herein. The resulting output imagery data 124 will have higherspatial resolution than previously attained with current state of theart methods, and (depending on system parameters) may also have betteramplitude-resolution and signal-to-noise ratio (SNR).

With reference again to FIG. 2, TDI imager system 100 may be configuredto operate as a readout integrated circuit with multi-accumulate-sampleselectable line rate capability by usage of the accumulate and shiftmodes. When operating in accumulate mode, controller 108 is configuredto read a value based on the stored charge from at least some of theunit cells 106 a,a-106 m,n. The accumulating storage locations 120a,a-120 m,n may each corresponds to a distinct one of the unit cells 106a,a-106 m,n. For example, storage location 120 a,a may correspond tounit cell 106 a,a, storage location 120 a,b may correspond to unit cell106 a,b, storage location 120 m,a may correspond to unit cell 106 m,a,and so on. When operating in accumulate mode, controller 108 may beconfigured to read and add a digital value (Co-Add data) based on thestored charge from unit cell 106 a,a to the existing value in storagelocation 120 a,a, add the value based on the stored charge from unitcell 106 a,b to the existing value in storage location 120 a,b, and soon. When operating in shift mode, controller 108 may shift the storagelocation for the Co-Added data digital values by one or more storagelocation positions prior to adding, and add the read the digital valueto a shifted storage location. For example, when the shift is equal toone pixel in the horizontal direction, the controller 108 may add thevalue based on the stored charge from unit cell 106 a,a to the existingvalue in storage location 120 a,b after shifting existing stored Co-Adddata from storage location 120 a,a to storage location 120 a,b, and soon. In these embodiments, the location that the Co-Add data is stored inmay be shifted along rows and/or columns of sensor 102 by a whole numberof pixels that is less than the m×n array size. A data output 130 may beloaded from storage location 120 m,a that corresponds to unit cell 106m,a and which contains Co-Add data that has been shifted over by one ormore corresponding pixels/detector areas in detector array 102.

Controller 108 may be configured to read a digital value based on thestored charge on less than all of the unit cells 106 during a particularsampling time. As scene 112 sweeps across the detector array 102, thedata-shifting mechanism of the unit cells 106 may be employed to movedigital values simultaneously with the movement of the scene. As Co-Adddigital data moves from an initial storage location 120 a,a to a readoutstorage location 120 a,n, it may or may not accumulate additional Co-Adddata along the way. When the readout storage location 120 a,n isreached, the data output 130 may be read out of the sensor 102.

In some embodiments, one or more input control signal(s) 116 may selectsome or all of the operational configuration parameters to beimplemented. For example, control signal 116 may indicate to controller108 whether to operate in accumulate or shift mode, which detectors 104and unit cells 106 may be active in accumulate mode, and/or theintegration time to allow each unit cell 106 to accumulate incidentscene photon signals. The control signal 116 may also indicate thenumber of unit cells 106 to accumulate in accumulate mode, the numberand addresses of storage locations 120 to shift in shift mode, the linerate, and/or the scan rate. The control signal(s) 116 may be operatorselected, or may be provided by external sensing or computing devices.In some embodiments, the number of pixel positions to shift the storagelocations 120 in shift mode may be determined based on the rate at whichan observed scene object is moving within the FOV of the array sensor102.

Evolved Enhanced MTF (EEMTF)

FIG. 3 is a flow diagram illustrating an EEMTF method 300 for generatingreduced scan smear line scan image data utilizing modified commanding asdescribed above, in accordance with some embodiments. Method 300 may beperformed by TDI imager system 100 operating in accumulate or shiftmodes.

In step 303, controller 108 may receive control signal 116 indicating aconfiguration for the detector array 102, and when to operate inaccumulate mode or shift mode, wherein enablement of each modecorresponds to certain readout-clock times. When indicating to operatein shift mode, the control signal 116 may also indicate the number ofstorage locations 120 to shift.

In step 304, the storage locations 120 of the digital memory structure118 may be cleared and capacitive elements of the unit cells 106 may bedischarged (e. g., reset) and charge may be allowed to accumulate on thecapacitive elements of the unit cells 106 over an integration timedetermined by controller 108.

In step 308, a determination is made by controller 108 whetheraccumulate mode is the current operational state for the imager system100. If so, controller 108 iteratively accumulates for a specifiedintegration time Co-Add data, for each read-out clock time while inaccumulate mode, and for each of a set of the unit cells 106 specifiedby controller 108, where each selected unit cell 106 has a PDFOV of thescene 112. In step 310, the digital (Co-Add) value is read from a unitcell 106 and is added in step 312 to the existing Co-Add data in thestorage location 120 corresponding to the unit cell 106. Steps 310 and312 are repeated (step 314) for at least some of the other unit cells106.

If, in step 308, the accumulate mode is determined to be disabled,logical operation passes to step 315, where a determination is made bycontroller 108 whether imager device 100 is to be operated in shiftmode. If shift mode is determined not to be enabled, controller 108returns processing to step 303 to execute additional signalaccumulation. If shift mode is enabled, controller 108 iterativelyperforms steps 318-320 for each selected unit cell 106 until the shiftmode is no longer enabled.

In step 318, the Co-Add data stored in the corresponding storagelocation 120 (e.g., data register) is shifted by the number of rowsbased on the rate of motion of the scene 112 in relation to the PDFOV(which may be specified by controller 108), in accordance with shiftmode as discussed above. Step 320 causes step 318 to be repeated for aplurality of the storage locations.

In step 324, the added Co-Add data from at least some of the storagelocations 120 are output as imaging data. The outputs may correspond toone line of data. At step 326, the EEMTF method 300 is repeated for thenext line of detectors.

As previously noted, line scanning detectors are continuously moved (orthe imaged object or scene 112 is moved across the sensor 102) and areconventionally sampled at a single high speed readout rate. Theover-sampled detectors 104 integrate energy while moving, and themismatch between the line scan rate and the readout rate may result inscan smear. Integration fraction may be defined as the time that aprojected detector (e.g., detector 104 a,a) is integrating energy versusthe time between data collects from an associated storage location 120a,a, the latter time being time during which data is being transferredand preparation for the next data collect occurs. During eachintegration period, the projected detector is moving relative to scene112, so a PDFOV of the scene 112 greater than the area of the projecteddetector is integrated to form a single pixel of image 128 of the scene112. The amount of blurring introduced by scanning is dependent on theintegration fraction, where a smaller integration fraction causes lessblurring, but also results in less acquired signal.

FIG. 4 illustrates with three exemplary scenarios 402, 404, 406 (for asingle detector) for aid in visualizing how slower line (output) ratesmay be achieved. Commands 408 are sent to the TDI detector array 102 ata constant readout rate that is independent of the line rate. Thereadout rate defines the maximum line rate of the detector(s) 104.Slower line rates may be achieved by temporally co-adding (oraggregating), in an accumulate mode (steps 310, 312, 314 described aboveabove), charge integrations of the detector(s) 104. For example,scenario 402 shows the typical operation of conventional scanningdetector arrays, where the line rate equals the readout rate and onecommand 408 a to collect data and send image data 410 a out is sent toeach detector(s) 104. In scenario 404, the line rate equals one-half thereadout rate, i.e., image data 410 b is sent out from the detector(s)104 on every other readout clock time (shown at times t2, t4, etc.).Commands 408 b includes instruction to aggregate data, but not to senddata out from the detector, while commands 408 c additionally instructthat image data 410 b be sent out. Scenario 406 presents the casewherein the line rate equals one-quarter the readout rate, achieved bythree accumulate mode commands 408 d data aggregations between eachimage data 410 c readout data command 408 e (at times t4, t8, etc.). Theintegration fraction may be selected such that a detector 104 does notcollect information for integration and addition to an accumulator forthe entire time of a readout clock time interval (e.g., t1 to t2). Forexample, if the integration fraction is 75% and the line rate equals ¼of the readout rate, during each time interval, the center of thedetector 104 moves a distance equal to ¼ of the detector area and iscollecting information for 75% of this time. For three readout clocktimes, the collected information is integrated and added to anassociated storage location 120. On every fourth readout clock time,data is selectively aggregated, and then the readout storage location120 sends out a single image data point.

With reference to FIG. 5, scan smear may be reduced through variabletemporal co-adding (accumulation), i.e., through selective over-sampleddetector integration. Through modified detector commanding, the numberof detector integrations co-added may be selectively reduced to produceeach set of image data. Scenario 502 shows the conventional processwherein the line-rate equals ¼ of the readout rate, and data is beingcollected and aggregated on each readout clock time. An effect ofutilizing every data sample available is larger effective pixels forlower line rates. Scenarios 504 and 506 show alternative methods whereincommands 508 a and 508 d instruct the detector(s) 104 to not collectCo-Add image data. Command 508 b comprises a collect data and aggregatecommand, and commands 508 c comprise data collect and send commands. Bytemporally co-adding fewer detector integrations (with or without areduced integration fraction) to produce each image data point 510 a,510 b, less of the scene 112 is imaged, thereby reducing the image scansmear by sampling less of the scene 112. While this results in adecreased number of scene photons being collected, the lost photons arefrom an undesired (blur causing) part of the scene 112 for the currentpixel being formed. So, while the image pixels for larger co-adding(larger blur) are formed from more photons, they do not represent thetrue scene as accurately as the low scan smear data scenario.

FIG. 6 depicts a portion 600 (i.e., two adjacent detectors 601, 603) ofdetector array 102 during exemplary detector integration stages overtime, utilizing four energy accumulation regions 602, 604, 606, 608 andan integration fraction of 100%. Co-Add region 602 extends for 25%longer than the in-scan height of the detector PDFOV 610. Co-Add regions604, 606, 608 accumulate energy in subsequent sampling periods shiftedby a ¼ PDFOV each time. The total area contributing energy to theaccumulation includes the initial PDFOV 610 through a quadratureextended PDFOV region 614 combined. A four Co-Add energy accumulationcentroid is depicted as detector-data-centroid 612. The resultantdetector-data-centroid 612 is based upon energy inclusive of two PDFOVs,which results in an in-scan smear component of one PDFOV.

FIG. 7 depicts an example embodiment where a single Co-Add accumulationPDFOV 700 of PDFOV 610 is shown for a single Co-Add energy accumulationregion set with an integration fraction of 100%. The Co-Add region 702extends for 25% longer than the in-scan height of PDFOV 610. The totalarea contributing energy to the accumulation includes initial PDFOV 610through a single extended PDFOV region 704. The resultant centroid of asingle Co-Add energy accumulation is depicted by detector-data-centroid706. The resultant detector-data-centroid 706 is based upon energyinclusive of a 1.25 PDFOVs, which results in an in-scan smear componentof ¼ PDFOV. The adjacent in-scan detector produces adetector-data-centroid 706 a that is one PDFOV in-scan offset.

FIG. 8 is a plot 800 demonstrating an improvement in scan smear MTFachieved in an example imager system 100 practicing the EEMTF scan smearreduction techniques described herein, as a function of line scanningfrequency. In this example, the line rate (1000 lines/sec) was equal to⅛ of the readout rate (125 lines/sec), and an integration fraction of0.88 was utilized. A greater improvement in MTF (i.e., differencebetween curve 802 resulting from an existing TDI line scan method andcurves 804 a and 804 b reflecting four Co-Add and single Co-Addembodiments of the EEMTF method 300, respectively) can be seen at higherfrequencies. More specifically, a 14.8% MTF enhancement at Nyquistsampling rate for the four sample embodiment (i.e., accumulating onlyfour middle samples of a line scan), and a 19.4% MTF enhancement for thesingle sample EEMTF embodiment, were demonstrated.

Analyses have shown that a scan smear reduction of greater than 20% maybe achieved in certain scenarios, and that the MTF may be maximallyimproved at the most difficult to achieve imaging scenarios.

FIGS. 9A and 9B are plot of experimental results, comparing scan smearresulting from an existing TDI line scan method and from four Co-Addembodiments of the EEMTF method 300. FIG. 9A reflects results fromimaging a detector pitch (P) of 1.0 bar target 902, and FIG. 9B a 1.5 Pbar target 904. In each case, reduced scan smear is evident in theplotted curves 906, 908 resulting from the EEMTF method, compared to theplotted curves 910, 912 resulting from the existing TDI line scanmethod. The conventional approach results in a smear of 1.985 pixels,whereas the four sub-sample EEMTF approach results in a 1.485 pixelsmear and the single sub-sample approach results in a 1.110 pixel smear.

In some embodiments, after a region of a scene 112 is scanned byadjacent TDI detector lines, the image data from each of those TDI linesmay be subjected to a range of image processing techniques. Matchingreference frames and adding the image data from the TDI lines mayachieve a higher resolution image. The co-add scene data may alsorequire digital scaling (e.g., in step 324 of FIG. 3) in order tomaximize the signal range of the imager device, i.e., to provide amaximally used output digital word. An enhanced resolution output imagemay also be constructed using techniques that will now be described.

Diagonal Imagery Collection Enhancement (DICE)

FIG. 10 is a flow diagram of a DICE method 1000 for constructing animage utilizing a TDI detector array 102 for any scan geometry and anyscan rate, and for processing the collected image data to producere-sampled imagery at various selectable effective resolutions.Embodiments of the DICE method 1000 permit the creation of images withthe various resolutions that are enhanced for over-sampled data sets andreduced for under-sampled data sets, while enabling image creationduring alternate imaging. A selective over-sampled detector integrationscan smear reduction capability somewhat similar to that described abovemay be employed in the DICE method 1000, but only optionally employingthe selective Co-Add capability of the imager system 100, andalternatively cross-scan oversampling with a diagonal system scan andoptionally an enhanced image product application employing up-samplinginterpolation. An innovative feature utilized in the DICE method 1000 isconfigurability of the TDI detector array 102 to reduce the array to asingle row, in order to enable usage of subset temporal data collectionfrom the over-sampled detectors in the diagonal scan mode to achieve amaximally sampled data set with virtually no diagonal scan smear.

Referring again to FIG. 1, a scene 112 (denoted in dashed lines) to beimaged is shown in motion relative to TDI imager system 100. In anexample embodiment, imaging system 100 is mounted such that the rows inthe detector array 102 form an angle to the direction of relative motionwith respect to the scene 112. Whereas in a conventional TDI imager thecamera would use all detector rows, in an example embodiment of the DICEconfigurable TDI imager, only a single row of detectors acquires scenesignal. Whereas in a conventional TDI imager the camera timing would besynchronized such that the image moves by exactly one pixel per frame,in an example embodiment of the DICE configurable TDI imager, the timingis set such that the scene 112 traverses any portion of a DPFOV betweenimage lines. In DICE methods 1000, controller 108 may be programmed tocompensate for the relative motion and summing on a high-resolutionimage grid as described below. Resulting output image data 130 will havehigher spatial resolution than previously attained with current state ofthe art methods, may (depending on system parameters) also have betteramplitude-resolution and signal-to-noise ratio (SNR), and may provideexpanded area coverage for a number of lines at lower resolution.

FIG. 11A depicts a portion 1100 (i.e., two adjacent pixel areas 1102,1104) of detector array 102 during exemplary detector integration stagesover time in an embodiment of a DICE method 1000. The depictedembodiment employs a single Co-Add energy accumulation region 1106 setwith an integration fraction of 100%. Co-Add region 1106 extends for thein-scan height of PDFOV 1108. The total area contributing energy to theaccumulation includes initial PDFOV 1108 through a single extended PDFOVregion 1110. The resultant centroid of a single Co-Add energyaccumulation is depicted by detector-data-centroid 1112. The resultantdetector-data-centroid 1112 is based upon energy inclusive of twoPDFOVs, which results in an in-scan smear component of one PDFOV.

With reference again to FIG. 10, DICE method 1000 may be performed by acombination of elements, such as controller 108 and other previouslydescribed components of imager system 100 (e.g., as shown in FIG. 2).DICE method 1000 may be performed for generating image data for scene112 in a combined sequence of optional accumulate modes and shift modes,each similar to the accumulate and shift modes described above.

In step 1005, an image data set 1002 is acquired in a manner similar tothe method shown in FIG. 3, but alternatively utilizing diagonalscanning, with the scan geometry and optionally the scan rate beingexternally controlled and the scan geometry reduced to a single row ofdetectors 104 and associated unit cells 106 of the sensor array 102 inan angled orientation relative to changes in relative position between aPDFOV of each selected detector 104 and the scene 112. Use of theaccumulate mode is also optional, in comparison to the method shown inFIG. 3.

In step 1010, a combination of the image data set 1002 acquired in step1005 and the associated scan geometry utilized to acquire the image dataset 1002 is stored as one or more small data set(s) 1004.

In step 1015, image data samples of the small data set(s) 1004 areformatted spatially into a spatially representative format, based ongeometric motion and sampling rate information contained in the scangeometry, for subsequent processing.

In step 1020, controller 108 receives (e.g., from an external sourcesuch as a user) image product parameters 1006 and one or more of thesmall data sets 1004. Controller 108 may then create (step 1020) andstore (step 1025) a large data set 1008 comprised of an output imageproduct 1012 in pixels and a geometric data set 1013 The output imageproduct 1012 may contain sub-regions of interest having selective higherresolution based on the received product parameters 1006. Thesub-regions of interest may be generated by processing (e.g., averaging,interpolating, normalizing, etc.) the spatially adjusted data setsamples of the small data set(s) 1004 associated with the sub-regions,as identified by the user image product parameters 1006. For example,data set samples of the output image product 1012 may be interpolatedbased on neighboring pixel image data sample values, a process known asresampling. The interpolation may be used to achieve “super resolution”,wherein the effective resolution of an image or a sub-region of an imagemay be increased. In some embodiments, super resolution is enabled byrepresenting the spatially representative format of the output imageproduct 1012 at a higher sample density than the original sample density(up-sampling). A high-resolution image grid may be constructed on whichto collect the data set samples of the output image product 1012. Thehigh-resolution image grid may be stored in computer memory (e.g., in amemory device included in and/or accessible to the controller 108).Pixels of the data set samples are assigned to pixels in thehigh-resolution pixel grid where the center of the high-resolution pixelmost closely represents the same image location as the center of thedata set sample pixels. When a subsequently generated output image mapsto the same pixels as an earlier output image, the high-resolutionvalues may be summed (increasing amplitude resolution), or the earliervalue may simply be replaced. If multiple values are summed, and whenthe number of values summed at all pixels may not be the same, thecontroller may keep track of the number of summed values so that thefinal pixel values can be normalized.

In step 1030, controller 108 may receive user image delivery parameters1014 that inform controller 108 whether the small data set(s) 1002and/or the large data set(s) 1008 are to be disseminated to one or moreuser(s). In step 1035, controller 108 transmits the requested data filesaccordingly.

In optional step 1040, the user may view and perform standard imageoperations on the disseminated small data set(s) 1002 and/or the largedata set(s) 1008. If the user received the small data set(s) 1002, theuser may select image product resolution and/or implement region ofinterest re-processing at selected resolutions, or implement region ofinterest re-processing of pre-defined areas at selected resolutions.

FIG. 11B depicts an example embodiment of a 2× resampling pixel grid1114, in which the detector array 102 is orientated at 45° in relationto the scan geometry (indicated by directional arrows 1115A and 1115B)with a scan rate of 0.707 of PDFOV 1116 per sample time increment, asdepicted by the spacing of the acquired data sample points (e.g., datasamples 1117 a through 1117 c). The resulting effective scan widthbecomes 0.707 of the original array scan width. Controller 108 mayevaluate interpolations (e.g., interpolation 1118 indicated by across-hatched dot) utilizing adjacent acquired data values 1117 athrough 1117 c. The 2× resampling high resolution pixel grid 1114 may begenerated from 0.5 PDFOV by 0.5 PDFOV image pixels aligned to a firstimage line. Acquired data 1117 b may be asserted at the center of a 1 Psquare 1119 (denoted by a white dot).

It is generally not the case that the acquired sample image data (fromsmall set(s) 1002) will fill into a high-resolution grid at one sampleper grid-point. FIG. 11C depicts another example embodiment, comprisinga 3× resampling pixel grid 1120, in which the detector array 102 isorientated at 45° in relation to the scan geometry (indicated bydirectional arrows 1122 a and 1122 b) with a scan rate equal to 0.707 ofPDFOV 1124, as depicted by the spacing of the acquired (small set) datasample points 1126. Controller 108 may similarly evaluate interpolations1128 utilizing values of adjacent acquired data samples (e.g., 1126 athrough 1126 c). The acquired data samples use the raw acquired data asit falls precisely upon a 3× image detector 104. The 3× resampling highresolution pixel grid 1120 may be generated from ⅓ PDFOV by ⅓ PDFOVimage pixels aligned to a first image scan line.

FIG. 11D depicts an additional example embodiment, this of a 4×resampling pixel grid 1140, in which the detector array 102 isorientated at 45° in relation to the scan geometry (indicated bydirectional arrows 1142 a and 1142B) with a scan rate equal to 0.707 ofPDFOV 1144, as depicted by the spacing of the acquired data samplepoints 1146 a through 1146 c. Controller 108 may similarly evaluateinterpolations 1148 utilizing adjacent acquired data values 1146 a,a1146 b, 1146 c. The 4× resampling high resolution pixel grid 1140 maybe generated from ¼ PDFOV by ¼ PDFOV image pixels aligned to a firstimage scan line.

FIGS. 11E and 11F depict additional example embodiments of 4× resamplingpixel grids 1160, 1180 where the detector arrays 102 are respectivelyorientated at 45° and at an acute angle in relation to the scan geometry(indicated respectively by directional arrows 1162 a-b and 1182 a-b),and with respective scan rates equal to 1.414 of PDFOV 1164 and greaterthan 1 PDFOV 1184, as depicted by the respective spacings of theacquired data sample points 1166 and 1186. As shown in FIG. 11E, asubset of high resolution pixels use duplicated pixels 1168 (in pixelgrid 1160) that are duplicates of the acquired data 1166 that areco-aligned, and interpolations 1169 and 1189 (in both 4× resamplingpixel grids 1160 and 1180). The 4× resampling high resolution pixelgrids may be generated from ¼ PDFOV by ¼ PDFOV image pixels aligned tothe first image line. Linear Delaunay triangulation based interpolationmay be used in some embodiments.

FIG. 12 depicts an example representation of an enhanced resolutionoutput (multi-resolution) image product 1200 including sub-regions 1202comprising low resolution image pixels 1204 and one or more sub-regionsof interest 1206 including high resolution image pixels 1208. The imageresolutions may be the same or different in each sub-region of interest1206.

FIG. 13 presents example comparative imaging results for diagonal TDIscanning of an original scene 1300 using conventional techniques (image1302) and using DICE processing (image 1304). The DICE enhancedprocessing parameters included use of a ¼ pixel grid (4×4 up-sampling).An improvement in resolution is clearly observable.

Simultaneous Secondary Mission Acquisition

In a secondary mission image acquisition embodiment, a super-resolutionTDI imager system 100 such as described above and shown in FIGS. 1 and 2maybe employed to construct an image of a desired image quality,additionally configured with a plurality of sensor arrays forindependently adjustable scan geometry and scan rates. A capability isprovided for simultaneous secondary mission acquisition during primarymission acquisition for the configurable multi-sensor platform. Aconfigurable secondary sensor or detector array may be utilized tocreate the secondary sensor content by modified controller commanding.An example scan geometry configuration may include reducing thesecondary detector array to a single row to achieve a sampled data setwith usable content limiting non-orthogonal scan smear, wherein thesampled data sets can be produced for any scan angle and scan rate lessthan or equal to 0.5 PDFOV per line rate sample.

FIG. 14A shows an exemplary four-sensor multi-mission system 1400scanned in a normal scan direction 1402 and at an angle scan vectors1404 a and 1404 b. Sensors 1406, 1408 and 1410 may or may not compriseconfigurable TDI unit cell arrays, but sensor 1412 comprises aconfigurable TDI unit cell array (e.g., configurable to a single row andhaving selective accumulation capability implemented to produce variablesmear detector samples) such as described above. In this embodiment,sensor 1412 comprises the configurable secondary sensor, arrangednon-parallel to a primary sensor array (sensor 1408 or 1410). Aspreviously described, each of the unit cells of sensor 1412 are adaptedto store charge based on detected photons, and are individuallyaddressable by a controller configurable to operate in accumulate andshift modes. Sensor 1412 also includes a digital memory structure havinga plurality of storage locations 120, with functionality and purposesimilar to those described above, where each storage location 120 isassociated with one of the unit cells 106.

While one of the primary sensors 1408 or 1410 acquires primary missionimage data for a scene 112, sensor 1412 simultaneously may acquiresecondary mission image data for the scene 112 scanning on anon-interference basis at various scan angles and scan rates with theprimary sensor array's image acquisition. Sensor 1412 non-orthogonallyscans the scene 112, during each sampling period, according to adiagonal scan geometry and in either an accumulate mode or a shift mode.During accumulate mode, for at least one of the sensor 1412 TDI unitcells 106 in the diagonal scan geometry, the controller 108 determines adigital value based on stored charge associated with the PDFOV, and addsthe determined digital value to the corresponding storage location 120.During the shift mode, for at least one of the sensor 1412 TDI unitcells 106 in the diagonal scan geometry, the controller 108 determines adigital value based on stored charge associated with the PDFOV of thescene 112, adds the determined digital value to the correspondingstorage location 120, and shifts stored summed digital values betweenadjacent corresponding storage locations 120. Sensor 1412 also generatessecondary image pixels for the scene 112 from the summed digital valuesof a set of the corresponding storage locations 120. The smear reductionand resampling techniques described above may additionally be utilizedin the multi-mission embodiments. Since a final high resolution imagemay be 16× as large as the original raw image data set, the option todisseminate to users the small data set(s) such as discussed above mayalso prove useful in the multi-mission application. Users may remotelycreate any of the image resolution output products from the small datasets.

The angled scan indicated by scan vectors 1404 a and 1404 b may beadjusted in scan angle and scan rate relative to sensor 1412 in order toachieve desired image quality independent of other sensors. Sensor 1406may be scanned at any of its normal rates using normal scan (indicatedby directional arrow 1402) while sensor 1412 is configured selectivelyfor image quality purposes and both sensors acquire data simultaneously.In some embodiments, sensors 1406, 1408 or 1410 may be configuredselectively, if available, for using angled scan vectors 1404 a and 1404b while sensor 1412 is configured selectively for image quality purposesand a plurality of the sensors acquire scene image data simultaneously.In various embodiments including capabilities similar to those describedabove, sensor 1412 may include a controller that selects the samplingrate for sensor 1412 based on changes in relative position between thePDFOV and the scene 112 due to a slew rate and geometry of the primarysensor 1406, 1408 or 1410. The diagonal scan geometry of sensor 1412 maybe configured to be a single row of the TDI unit cells 106 of sensor1412 in an angled orientation relative to direction of scene motion.

FIG. 14B illustrates an example embodiment of a three-sensor 1430, 1432,1434 multi-mission system 1400 scanned in a normal to sensor 1430. Inthis embodiment, each of sensors 1430, 1432 may or may not compriseconfigurable TDI unit cell array 102, whereas sensor 1434 is aconfigurable TDI unit cell array 102. The normal scan direction tosensor 1430 is depicted by directional arrows 1436 a, 1436 b, and 1436c. Sensor 1430 may be scanned at any of its normal scan rates, whilesensor 1434 is configured selectively for image quality purposes, andboth sensors 1430, 1434 may acquire data simultaneously. Sensor 1432 maybe configured selectively, if available, for image quality purposes, andall three sensors acquire data simultaneously. Sensor 1434 may beconfigured selectively for a normal acquisition.

The detector array of sensor 1432 may be configured into a sparsepattern of active detectors, so that image data sets can be produced fora normal scan angle and a selected scan rate slower or faster than aPDFOV per line rate sample. For any scan rate, the controller 108 cansmear any number of PDFOVs in the creation of an image data set. Thesparse pattern for high scan rates enables sparse sampling to reducetotal number of data samples, while maintaining a balance betweenin-scan and cross-scan MTF. FIG. 15 depicts an example embodiment of asparse enabled configurable TDI unit cell array 1500, wherein a normalscan vector 1502 is utilized for image generation at a 4×PDFOV scanrate. In this example, only one sample (acquired by active detector1504) in each four sampling periods is accumulated and shifted, and witha multitude of disabled detectors 1506 per TDI scan column 1508. Thereduced sampling decreases the array output data rate, as discussedabove.

FIG. 16 depicts an example embodiment of a sparse array interpolatedimage 1600 projected on ground over time (i.e., white blocks 1620representing non collected PDFOV; sampling four locations) for sparseenabled configurable TDI unit cell array sensor 1432, where a normalscan vector 1610 is utilized for image creation at a 4×PDFOV scan rate.In this embodiment, only one sample of each four sample periods is beingaccumulated and shifted of the PDFOV 1615, yielding a centroid of thesecondary sensor scanned detector based upon a scanned detector regionof integration 1640 set with a final image being constituted ofinterpolated image pixels 1630 (the pixel smear sample centroid).

The multi-mission embodiments overcome the limitations of conventionalfield offset, non-parallel multi-sensor platforms that acquire only asingle sensor's data per collect, as the other TDI array sensors arelimited to orthogonal scanning, and thus can obtain no usable imageryfrom conventional secondary sensors in pre-existing operating modes.FIG. 17 presents images resulting from a multi-mission imagingexperiment wherein test system parameters included: a line scan rate of1000 lines/second; an integration fraction of 88%; a readout rate of1000 lines/second; pixel spacing of 1.0 pitch cross and in-scan; aneffective array scan angle of 60° from orthogonal at 1 pixel slew rate(secondary sensor rotated 120° from the primary sensor); and a smear of1.88 pixels of a rotated detector (i.e., 1 P+P*88% integrationfraction). Image 1700 comprises the original scene image, image 1702 theconventionally processed (non-simultaneously acquired image), and image1704 the multi-mission simultaneously acquired super-resolution image,resampled to a 4× high resolution pixel grid.

Whereas many alterations and modifications of the disclosure will nodoubt become apparent to a person of ordinary skill in the art afterhaving read the foregoing description, it is to be understood that theparticular embodiments shown and described by way of illustration are inno way intended to be considered limiting. No element, act, orinstruction used herein should be construed as critical or essentialunless explicitly described as such. Where only one item is intended,the term “one” or similar language is used. Also, as used herein, theterms “has”, “have”, “having”, or the like are intended to be open-endedterms. Further, the phrase “based on” is intended to mean “based, atleast in part, on” unless explicitly stated otherwise.

What is claimed is:
 1. A method of generating reduced in-scan blur imagedata using an imaging device including a controller, a digital memorystructure having a plurality of storage locations, each storage locationto store a digital value and being individually addressable by thecontroller, and an array of unit cells configured to store charge basedon detected photons, each of the unit cells being associated with acorresponding storage location and including a time delay and integrate(TDI) detector, the imaging device having a readout rate that is greaterthan a line rate, the readout rate defining readout-clock times, themethod comprising the steps of: selecting shift mode readout-clock timesand accumulate mode readout-clock times, the accumulate modereadout-clock times representing times during which scene data should beaccumulated from one or more of the unit cells; for each of the selectedaccumulate mode readout-clock times and for each of a set of the one ormore unit cells: accumulating over an integration time detector chargeassociated with a projected detector field of view (PDFOV) of the sceneto obtain a detector co-add data sample, reading the detector co-adddata sample from the unit cell, adding the detector co-add data sampleto existing detector co-add scene data in the storage locationcorresponding to the unit cell; and for each of the selected shift modereadout-clock times, and for each unit cell in the set of one or moreunit cells: shifting the existing detector co-add scene data stored in aplurality of the storage locations corresponding to the set of unitcells to a plurality of shifted storage locations by a number of rowsthat is based on a rate of motion change of the scene in relation to thePDFOV, and outputting at the line rate from one or more of the pluralityof storage locations the detector co-add scene data corresponding to aline of scene data, to form an enhanced image of the scene.
 2. Themethod of claim 1, further comprising selecting the line rate.
 3. Themethod of claim 1, further comprising selecting the integration time. 4.The method of claim 1, further comprising selecting for accumulating theset of the one or more unit cells.
 5. The method of claim 1, furthercomprising selecting which one or more unit cells may be active duringthe selected accumulate mode readout-clock times.
 6. The method of claim1, wherein the data sets are captured in visible and infrared wavelengthranges.
 7. The method of claim 1, wherein: the TDI detectors included inthe array of unit cells are arrayed in one or more rows; and therelative position of the scene changes a fractional portion or anintegral number of PDFOVs along an in-scan direction of the one or morerows of the detectors between each successive readout-clock time.
 8. Themethod of claim 1, further comprising selecting the data shift instorage locations for the detector co-add scene data so as to maximizethe signal range of the imaging device.
 9. A method of generatingreduced in-scan blur image data with an imaging device including anarray of unit cells configured to store charge based on detectedphotons, a controller, and a digital memory structure having a pluralityof storage locations, each storage location to store a digital value andbeing individually addressable by the controller, each of the unit cellsbeing associated with a corresponding storage location, the methodcomprising the steps of: determining a digital value based on a storedcharge from at least one of the unit cells; adding the determineddigital value to an existing value in a corresponding storage locationwhen operating in accumulate mode; and selectively adding the determineddigital value to an existing value in a storage location when operatingin accumulate mode; wherein during shift mode said storage locationsshift values from adjacent storage locations and an output is generatedfrom a set of storage locations.
 10. A line scanning imaging devicehaving a readout rate that is greater than a line rate, the readout ratedefining readout clock times, comprising: a focal plane array of unitcells each including a time delay and integrate (TDI) detectorconfigured to accumulate charge based on detected photons; a digitalmemory structure having a plurality of storage locations, eachassociated with a corresponding unit cell and configured to store adigital value indicative of the charge and to be individuallyaddressable; and a controller is configured to operate in an accumulatemode and a shift mode; wherein, in the accumulate mode, the controllerdetermines the digital value based on the stored charge from at leastone of the unit cells, and selectively adds the at least one determineddigital value to existing values in at least one corresponding storagelocation, and in the shift mode, the controller shifts values from theat least one corresponding storage location to adjacent storagelocations.
 11. The line scanning imaging device of claim 10, wherein thecontroller is further configured to: select shift mode readout-clocktimes and accumulate mode readout-clock times, the accumulate modereadout-clock times representing times during which scene data should beaccumulated from one or more of the unit cells; for each of the selectedaccumulate mode readout-clock times and for each of a set of the one ormore unit cells, accumulate over an integration time the detector chargeassociated with a projected detector field of view (PDFOV) of the sceneto obtain a detector co-add data sample, read the detector co-add datasample from the unit cell, add the detector co-add data sample toexisting detector co-add scene data in the corresponding storagelocation; and for each of the selected shift mode readout-clock times,and for each unit cell in the set of one or more unit cells, shift theexisting detector co-add scene data stored in the at least onecorresponding storage locations to a plurality of shifted storagelocations by a number of rows that is based on a rate of motion changeof the scene in relation to the PDFOV, and output at the line rate fromone or more of the plurality of storage locations the detector co-addscene data corresponding to a line of scene data, to form an enhancedimage of the scene.
 12. The line scanning imaging device of claim 10,wherein the TDI detectors are responsive to electromagnetic radiation inthe visible and infrared regions of the spectrum.
 13. The line scanningimaging device of claim 10, wherein the array of unit cells have animaging geometry providing that the scene is repositioned along anin-scan direction of the array.
 14. The line scanning imaging device ofclaim 10, wherein the controller is further configured to select theline rate.
 15. The line scanning imaging device of claim 11, wherein thecontroller is further configured to select the integration time.
 16. Theline scanning imaging device of claim 11, wherein the controller isfurther configured to select for accumulating the set of the one or moreunit cells.
 17. The line scanning imaging device of claim 11, whereinthe controller is further configured to select which unit cell may beactive during the selected accumulate mode readout-clock times.
 18. Theline scanning imaging device of claim 10, wherein the unit cellscomprise charge-coupled devices.